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ABSTRACT 

The assertion that there are a limited set of generalizable good educational practices (Chickering & 
Gamson, 1987) with a common model of preferred active student engagement in learning (Kuh, 2001) is 
appealing to those responsible for simply stated institutional outcomes and to the faculty who teach in 
fields that espouse the same practices and outcomes (Braxton, 1998). After all, if they are wrong and 
educational experience and good educational practices differ in important, substantive, and replicable 
ways by area of academic major, then assessment, accountability, administration, and admissions 
become more complicated and less amenable to central “oversight” and uniform standards. This paper 
reports that there are indeed important differences in student experience and engagement by academic 
discipline, that disciplinary patterns of student experience cluster, and that academic performance by 
students in these clusters is differentially predicted by standard admissions measures and student 
engagement factors. Recognizing the differences and identifying the predictors will lead to better 
admission practices. 



In 2001 , the University of California embarked on an effort to increase by an order of magnitude 
the resolution with which it could perceive the undergraduate experience through survey 
activities by asking all students at all campuses to answer both a series of common questions 
and a randomly assigned supplemental module. Its efforts continued and evolved and in 2008, 
over 60,000 students completed the Student Experience in the Research University’s (SERU’s) 
University of California Undergraduate Experience Survey (SERU/UCUES), a 40% response 
rate. 



* The SERU Project and Consortium is a collaborative of 15 major research universities based at the Center for 
Studies in Higher Education at UC Berkeley and including the administration of the SERU survey of undergraduates. 
A version of this paper was presented at the conference “Beyond the SAT: Rethinking Admissions” at Wake Forest 
University, April 15 and 16, 2009. 
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These data were confidential, requiring a student login, and therefore, the results can be linked 
with all other institutional records. Given a common core set of items, four or five randomly 
assigned modules, and the ability to link to admissions, registration, and financial aid files, this 
creates endless opportunities for policy studies and academic research. One highly successful 
application of these data has been comparative academic program review, where comparisons 
across campuses can be made at the level of the academic major. For example, responses by 
political science majors at Berkeley can be compared to those of political science majors at 
UCLA and San Diego or to any combination of campuses. 

The results differ from intra-campus major to college or major to campus comparisons in ways 
that lead to different conclusions and interventions. To have so many options for inquiry 
supported by a regularly administered comprehensive questionnaire that is supplemented by 
institutional operating system records can be somewhat overwhelming. Especially when applied 
to a practical matter like deciding which applicants to admit, one challenge is to reduce 
complexity without blurring necessary detail. 

So, what detail is necessary to consider when creating and applying admissions policies? 

This paper argues that cumulative academic performance is a preferred outcome over general 
skills acquisition and that there is, in fact, no such thing as general educational skills acquisition. 
Stated simply, “general” education experience is not uniform across disciplines and, therefore, 
the extent to which performance can be predicted by common admissions measures varies 
widely. 

More importantly, general education outcomes are not of much interest to higher education 
constituencies, with the notable exception of some administrators and many governmental 
bureaucrats. Students, parents, faculty, and employers are primarily concerned with outcomes 
in academic disciplines, and faculty additionally desire to teach appropriately engaged students. 

The Myth of a General Education 

There are several bodies of evidence to support the assertion that general educational 
outcomes can accurately be measured only within the context of academic disciplines. While 
there are skills developed in college that cross disciplinary areas, all are expressed within the 
epistemology of the discipline. 

These bodies of evidence have been amassed by researchers using a variety of methods, but 
one trend is clear: Researchers who have used methods capable of testing for disciplinary 
differences in general skill acquisition have found them. If general educational outcomes reflect 
disciplinary differences, then it is logical to assume that the variables that predict performance 
will also differ by discipline. 

Relevant Literature 

Among the bodies of evidence that general skills outcomes reflect disciplinary differences, eight 
will be briefly summarized. 



CSHE Research & Occasional Paper Series 




Chatman, DISCIPLINARY PATTERNS 



3 



Donald 

Janet Gail Donald has led a series of studies into epistemological differences in disciplines and 
how they are conveyed to students: What is valued? How is knowledge advanced? How is 
knowledge transmitted? These are questions answered differently in different academic fields. 
Her methodology has been anthropological — relying on interviews, observation, and 
exemplars. It began when she dropped the assumption that all teaching and learning is uniform 
across disciplines and decided in 1976 to understand those processes in other disciplines. Her 
2002 book, Learning to Think: Disciplinary Perspectives, is a persuasive accounting of her work. 

Beyer 

Catherine Beyer and colleagues at the University of Washington employed a variety of 
quantitative and qualitative methodologies in completing a four-year longitudinal study of 
undergraduate students’ personal, social, and intellectual development. As reported in Beyer, 
Gillmore, and Fisher’s Inside the Undergraduate Experience (2007), they concluded that all 
academic learning is mediated by the discipline, even in courses labeled “general education.” 
Moreover, the disciplinary mediation begins in lower-division coursework. “We have learned, 
and we argue here, that there is no such thing as an undergraduate education; instead, we have 
many undergraduate educations filtered through the lenses of particular disciplines ...” (p. 23). 

Smart 

With several colleagues over a number of years, John Smart has convincingly demonstrated 
that disciplines, faculty, and students’ behaviors and attitudes can be described by John Holland 
types and that students do best in compatible disciplinary fields in the same way that employees 
are most successful in compatible work environments. Holland types are realistic, investigative, 
artistic, social, enterprising, and conventional. Moreover, commonalities among types by 
discipline are reflected in academic organization structures. An excellent summary of the work is 
Smart, Feldman, and Ethington (2000), Academic Disciplines: Holland's Theory and the Study 
of College Students and Faculty. 

Biglan 

Similar to the atheoretical, empirically based classification strategy used in this paper, Biglan 
derived a three-dimensional solution using multidimensional scaling of faculty ratings of subject 
matter area similarities. The dimensions of hard/soft, pure/applied, and life/nonlife responses 
have proven to be remarkably useful to higher education researchers because they have been 
shown to distinguish everything from faculty attitudes and behaviors to class size. His 1973 
Journal of Applied Psychology papers are a good starting point. 

Nelson-Laird, Kuh, Pike, and others 

Whether viewed from the perspective of student learning behaviors (2005) or faculty teaching 
behaviors (2006), cross-institutional studies from the National Survey of Student Engagement 
(NSSE) and Faculty Survey of Student Engagement (FSSE) show reliable and consistent 
disciplinary differences in information processing that tend to favor social sciences and 
humanities over sciences and engineering. In their 2002 publication, Gary Pike et al. described 
how disciplinary differences affected institutional scores and thereby comparisons or rankings 
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based on those scores. Put simply, institutional measures and relative institutional scores may 
reflect little more than program composition. 

Chatman 

The idea that there can be more variance within campus by discipline than by discipline 
between campuses was pursued by Chatman in 2006. Using the eight-campus census survey, 
University of California Undergraduate Experience Survey (UCUES), Chatman used factor 
scores to cluster programs based on similarities. The outcome was a sorting that tended to 
replicate academic structures — at least suggesting construct validity. In addition, the pattern of 
factor scores by cluster tended to replicate the results of NSSE researchers described earlier 
with humanities and social sciences students scoring higher on typical measures of 
engagement and sciences and engineering students scoring lower. 

The UCUES principal component factor solutions did find quantitative and research 
engagement factors where the disciplinary patterns were reversed. Chatman’s paper showed 
how AAU institutions would fare in rankings based solely on mix of programs. Interestingly, 
traditional engagement scores placed perennially top-ranked Harvard and Yale at the top of 
ranked lists based on nothing more than program composition. 

Brint, Cantwell, and Hanneman 

In their Research in Higher Education article “The Two Cultures of Undergraduate Engagement” 
(2008), Steve Brint, Allison Cantwell, and Bob Hanneman described two cultures based on 
student responses to a subset of 15 items measuring student engagement from the 2006 
University of California Undergraduate Experience Survey (UCUES). The humanities/social 
sciences culture was identified by differences in interaction, participation, and interest in ideas. 
The natural sciences/engineering culture valued improvement of quantitative skills, collaborative 
study, and career orientation. 

Academic majors could be sorted into one of the two cultures and among cultural differences 
were post-graduate degree plans, social class and gender, and academic aptitude. Given that 
there were two distinct cultures of academic engagement, the authors asserted that models of 
good practices in education and student engagement in learning that assumed a common, 
“normative” view applicable to all students in all majors would not likely be embraced by both 
cultures. 

Arum and Roska 

The four-year longitudinal study of Collegiate Learning Assessment (CLA) involving over 2,300 
students attending 24 institutions supported disciplinary differences. Arum and Roska (2008) 
reported that general education outcomes as measured by the CLA were affected by academic 
major. “Our analyses confirm the relevance of college major. Students majoring in science and 
math as well as those majoring in social sciences and humanities exhibit higher growth in 
cognitive skills, as measured by the CLA, than students majoring in business. Students majoring 
in engineering, agriculture and computer science also experience more cognitive growth, 
although of smaller magnitude” (p. 1 1 ). 
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As is consistent with the work of the other authors, they noted that there were fields more 
conducive to the acquisition of cognitive skills as measured by the CLA: critical thinking, 
analytical reasoning, and written communication. Braxton and others have labeled those 
academic areas affinity disciplines. 

Methodology 

There were two analytical steps used in this study. First, previously determined student factor 
score patterns (Chatman, 2008) were subjected to cluster analysis to determine naturally 
occurring similar response profiles for academic majors. Resulting academic clusters became 
the highest level of defensible aggregation for predictive validity studies. 

Second, the relative strength of various predictive measures was determined using SAS 
GLMSELECT where the dependent variable was cumulative grade point average of upper- 
division majors — a traditional predictive validity design with less traditional predictors and 
performed at the level of disciplinary clusters. 

Factor Structure 

The factor structure of the first University of California System survey of all undergraduates, 
SERU/UCUES 2006, was determined by a research team using principal components analysis 
with varimax rotation followed by promax rotation within principal components. The process and 
results were described in Chatman, 2007. 

The intercampus UCUES Institutional Research Work Group made several changes to the 2006 
core segment of the questionnaire for 2008 based on feedback from students and the results of 
various research projects conducted in 2006 and 2007. Student use of time, campus climate 
and diversity, and academic experience components were increased or moved from randomly 
assigned modules to the common core. 

A new factor analysis was therefore in order and the results were described elsewhere 
(Chatman, 2008). The 2008 solution replicated the 2006 methodology and the results were 
exactly the same for four factors and very nearly for a fifth. There were three new principal 
components: Factors 3, 6, and 7. The 2008 solution was comprised of eight factors: 

• Factor 1 : Satisfaction with Educational Experience 

• Factor 2: Current Skills Self-Assessment (Nonquantitative) 

• Factor 3: Engagement with Studies 

• Factor 4: Gains in Self-Assessment of Skills (Nonquantitative) 

• Factor 5: Development of Scholarship 

• Factor 6: Campus Climate for Diversity 

• Factor 7: Academic Disengagement 

• Factor 8: Quantitative Professions 
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Cluster Analysis 

Student scores on these eight factors and academic time (time studying and in class) were 
computed and subjected to cluster analysis for academic major using an agglomerative method. 
The analysis was performed at the level comparable to a two-digit CIP Code. The resulting 
dendogram is shown as Figure 1. The vertical line marks a useful and reasonable level of 
centroid separation that sorts disciplinary areas into nine clusters. (The numbers following 
disciplinary area labels are University of California area codes.) 

Figure 1: Empirically Derived Academic Discipline Clusters 
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The empirically derived academic structure for a hypothetical university formed from the eight- 
campus University of California composite had nine divisions: 

1. Social Sciences — Interdisciplinary Studies, Social Sciences, Psychology, 
Communication, Law and Consumer Sciences 

2 . Ag ricu Itu re — Ag ricu Itu re 

3. Fine Arts — Fine Arts 

4. Humanities — Foreign Languages and Letters 

5. Public Administration — Area and Ethnic Studies and Public Administration 

6. Architecture and Environmental Design — Architecture and Environmental Design 

7. Science and Math — Biological Sciences, Physical Sciences, Mathematics, Computer 
Science 

8. Engineering — Engineering 

9. Business and Management — Business and Management 

A similarly organized real university would not be exceptional. In fact, except for the placement 
of Public Administration (Area and Ethnic Studies and Public Administration), it would be fairly 
typical. Recall again that the analysis began with student scores and found disciplinary clusters. 
It did not begin with disciplinary clusters and then look for confirmation. The analysis made no 
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assumptions about student experience within academic disciplines. The resulting academic 
clusters were used subsequently as a reasonable level of aggregation for undergraduate 
academic outcomes — all subsequent analyses were performed at the level of these clusters. 
The next analytical step was to determine the variables that were useful predictors of academic 
performance within the clusters. 

Factors Associated with Academic Performance 

Seven of the SERU/UCUES 2008 factors were considered in the prediction of cumulative grade 
point average in addition to important demographic variables and standard admissions 
measures collected by the university. Given that the California Constitution prohibits 
consideration of sex, race, or ethnicity in public university admissions, the demographic 
variables included were first-generation college (parents without bachelor degree), English as a 
first language (not other language or combination of other language and English), family income 
(above or below $65,000), and the Academic Performance Index score (API) quintile rank for 
high school attended. The standard measures included were high school grade point average 
and SAT Reading/SAT1 Verbal and SAT Math/SAT1 Math and SAT Writing/SAT2 Writing\ The 
factors and the items that comprise these factors are shown in Appendix 1 . 

Analysis 

To avoid known problems of stepwise regression bias (Flom & Cassell, 2007), SAS Procedure 
GLMSELECT was used. The forward choice fit statistic specified was the Schwarz Bayesian 
Information Criterion (Schwarz, 1978) that balances increased prediction from the addition of 
new variables with a penalty term for number of parameters. In addition, students were limited 
to upper-division declared majors who matriculated as freshmen, and the declared majors had 
to fit the clusters identified in the first step. 

Table 1: Descriptive Statistics for Variables Used in Analysis (N=15,171) 





Mean 


Standard 

Deviation 


Academic Performance 


Cumuiative GPA (4.0 maximum) 


3.21 


0.44 


Principal Component Factors 


Engagement Factor* 


5.2 


2.0 


Disengagement Factor* 


5.1 


1.8 


Quantitative Factor* 


5.0 


2.0 


Academic Time Factor* 


4.9 


1.9 


Fiigh Schooi Measures 


FIS Grade Point Average (Unweighted) 


3.61 


0.31 


SAT Math/SAT1 Math 


634 


88 


SAT Reading/SAT1 Verbai 


600 


91 


SAT Writing/SAT2 Writing 


608 


94 


Academic Performance Index Quintile 


4.3 


1.7 


FIS Flonors Classes 


15 


7.1 



* standardized to a mean of 5 and standard deviation of 2 
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The group with complete data records was described in Table 1. The average University of 
California freshman matriculants reaching the upper division were well prepared for college. For 
the upper-division freshman matriculants in this study, mean unweighted high school GPA was 
3.61, and SAT scores were 634 (SAT Math/SATI Math), 600 (SAT Reading/SATI Verbal) and 
608 (SAT Writing/SAT2 Writing). They had also taken 15 high school honors classes on 
average. Across the University of California, six-year graduation rates were about 80%. 



Results 

Stepwise optimal solution produced variable series with two important characteristics for this 
paper. First, the variables selected were intuitively appealing and consistent with prior research 
(Table 2). For example, the first SAT score entered was Writing (SAT Writing/SAT2 Writing) for 
Social Sciences, Fine Arts, and Business Management; Reading (SAT Reading/SATI Verbal) 
for Agriculture, Language and Foreign Language; and Math (SAT Math/SATI Math) for 
Mathematics and Science and Engineering. It is also interesting that SAT Math/SATI Math was 
not included for Social Sciences, Fine Arts, Letters and Foreign Language. 

These results confirm notions about skills important to or developed by instruction in academic 
areas and offer construct validity. The second important characteristic for this paper was that 
questionnaire-based factors frequently entered into the predictive series, especially 
Engagement, Disengagement, and Quantitative Professions for Mathematics and Science, 
Engineering, and Business Management. Self-ratings of skill gains were occasionally added, but 
not uniformly. 

Table 2: Relative Predictive Strength by Academic Discipline Area 





Social Sciences 


Agriculture 


Fine Arts 


Letters & Foreign 
Language 


Mathematics & 
Science 


Engineering 


Business Management 






Predicting Cumulative Grade Point Average of Upper-Division Students Matriculating as Freshmen 




1st 


SATWriting/SAT2 Writing 


HSGPA 


SATWriting/SAT2 Writing 


1st Language 


HS GPA 


HS GPA 


SAT Writing/SAT2 Writing 


2nd 


HS GPA 


SAT Reading/SATI Verbal 


HS GPA 


Engagement 


SAT Math/SAT1 Math 


SAT Math/SATI Math 


HSGPA 


3rd 


Engagement 


Disengagement 


Engagement 


Disengagement 


Engagement 


Engagement 


Engagement 


4th 


Disengagement 


SAT Math/SATI Math 


Disengagement 


HSGPA 


SAT Reading/SATI Verbal 


Disengagement 


SAT Math/SATI Math 


5th 


SAT Reading/SATI Verbal 




1st Generation 


SAT Reading/SATI Verbal 


Disengagement 


SATWriting/SAT2 Writing 


D/sengagement 


6th 


HS API 




HSAPI 


SAT Writing/SAT2 Writing 


Skill Gains (NonQuant) 


Skill Current (NonQuant) 


Quant Professions 


7th 


Skill Gains (NonQuant) 








Skill Current (NonQuant) 


Quant Professions 


HSAPI 


8th 


Skill Current (NonQuant) 








Academic Time 


Dev. of Scholarship 




9th 


1st Generation 








Quant Professions 






10th 


1st Language 








SAT Writing/SAT2 Writing 






11th 










HS API 






12th 










Dev. of Scholarship 








0.36 


0.34 


0.35 


0.47 


0.39 


0.31 


0.36 


N 


6,643 


418 


606 


866 


3,954 


1,623 


669 



To illustrate the extent to which disciplinary variation was important in describing the 
relationships, correlations among four factor scores, three SAT scores, high school GPA, and 
cumulative GPA were presented as Table 4. Engagement was typically correlated about 0.20 
with cumulative GPA but not in Agriculture. Disengagement was always negatively correlated 



’ If an upper-division student in spring 2008 had submitted SAT scores after the addition of SAT Writing, then SAT 
Critical Reading was treated as SAT 1 Verbal and SAT2 Writing as SAT Writing. 
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with GPA but less strongly in Mathematics and Science and Engineering where curricular 
practices differed from those in Social Sciences, Humanities, and Agriculture. 

The strongest simple correlations for the Quantitative Professions factor were negative 
correlations for Letters and Foreign Languages, and Fine Arts. It is also important to note that 
the principal component factors were no more than weakly correlated with SAT scores and were 
nearly always more strongly correlated with GPA than with SAT scores. The correlations by 
discipline between Engagement or Disengagement and SAT were about 0.10 or less and 
frequently less than 0.05. 

The correlations between Quantitative Professions and SAT scores were surprisingly negative. 
And last, considering the two most frequently used admissions measures, both high school GPA 
and SAT scores were consistently moderately to strongly correlated with cumulative GPA. High 
school GPA was not always the most highly correlated. 



For the seven disciplinary areas shown in Table 4, there were three ties, three times that an 
SAT score was more highly correlated, and once when high school GPA was more highly 
correlated. The SAT score that was most highly correlated was more closely aligned with the 
content of the discipline: 



o Social Sciences 
o Agriculture 
o Fine Arts 

o Letters and Foreign Languages 
o Mathematics and Sciences 

o Engineering 
o Business and Management 



SAT Reading/SATI Verbal 
SAT Writing/SAT2 Writing 
SAT Writing/SAT2 Writing 
SAT Writing/SAT2 Writing. 

SAT Math/SAT 1 math and SAT 
Writing/SAT2 Writing 
SAT Math/SATI Math 
All Three Scores 



Viewed collectively, the results clearly supported the inclusion of Writing (SAT Writing/SAT2 
Writing) as an SAT component. 



The factor scores from SERU/UCUES 2008 were contemporaneous with cumulative grade point 
average and were therefore not necessarily causal. Did engaged students succeed at higher 
rates or does academic success encourage student engagement? The answer was not 
available from this study but should become addressable with subsequent questionnaire 
administrations, including the extension of this line of questioning to incoming freshmen. 

An indirect indicator was available. If Engagement within academic discipline was strongly 
correlated with admissions test scores, then there would be support for the idea that good 
students were more engaged and got better grades. The GLMSELECT results for Engagement 
as a dependent variable were also in Table 2. 

From these lists it was apparent that admissions test scores were infrequently associated with 
Engagement and that demographic variables dominated. More importantly, variance explained 
(R^) for the models never exceeded 3%. Predicting student engagement with reasonable 
accuracy will require something more. 
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Table 3: Correlations Between Select Factor Scores, Admissions Scores and Cumulative GPA 





GPA 


Engagement 


Disengag 

ement 


Quantitative 

Professions 


SAT 

Academic Reading/ 

Time HS GPA SAT1 Verbal 


SAT 

Math/ 

SAT1 

Math 


SAT 

Writing/ 

SAT2 

Writing 


GPA 


1.00 


0.18 


-0.19 


-0.05 


Social Sciences 
0.09 0.40 


0.43 


0.31 


0.43 


Engagement 


0.18 


1.00 


-0.06 


0.13 


0.27 


■0.02 


0.00 


-0.08 


0.01 


Disengagement 


-0.19 


-0.06 


1.00 


-0.05 


■0.16 


- 0.09 


0.03 


0.10 


0.03 


Quantitative Professions 


-0.05 


0.13 


-0.05 


1.00 


0.13 


- 0.04 


- 0.14 


0.00 


■0.12 


Academic Time 


0.09 


0.27 


- 0.16 


0.13 


1.00 


0.07 


- 0.05 


- 0.01 


■0.05 


HS GPA 


0.40 


-0.02 


- 0.09 


- 0.04 


0.07 


1.00 


0.26 


0.26 


0.29 


SAT Reading/ SAT1 Verbal 


0.43 


0.00 


0.03 


-0.14 


- 0.05 


0.26 


1.00 


0.59 


0.79 


SAT Math/ SAT1 Math 


0.31 


-0.08 


0.10 


0.00 


■0.01 


0.26 


0.59 


1.00 


0.61 


SAT Writing/ SAT2 Writing 


0.43 


0.01 


0.03 


-0.12 


■0.05 


0.29 


0.79 


0.61 


1.00 


GPA 


1.00 


0.03 


-0.21 


-0.02 


Agriculture 
0.13 0.41 


0.41 


0.33 


0.40 


Engagement 


0.03 


1.00 


0.03 


0.10 


0.18 


■0.02 


0.06 


-0.11 


0.11 


Disengagement 


-0.21 


0.03 


1.00 


-0.01 


■0.17 


■0.07 


0.04 


0.03 


■0.03 


Quantitative Professions 


-0.02 


0.10 


-0.01 


1.00 


0.02 


■0.13 


■0.20 


-0.07 


■0.22 


Academic Time 


0.13 


0.18 


-0.17 


0.02 


1.00 


0.11 


■0.07 


-0.07 


■0.04 


HS GPA 


0.41 


-0.02 


-0.07 


-0.13 


0.11 


1.00 


0.19 


0.21 


0.21 


SAT Reading/ SAT1 Verbal 


0.41 


0.06 


0.04 


-0.20 


■0.07 


0.19 


1.00 


0.49 


0.80 


SAT Math/ SAT1 Math 


0.33 


-0.11 


0.03 


-0.07 


■0.07 


0.21 


0.49 


1.00 


0.46 


SAT Writing/ SAT2 Writing 


0.40 


0.11 


-0.03 


-0.22 


■0.04 


0.21 


0.80 


0.46 


1.00 


GPA 


1.00 


0.17 


-0.26 


-0.11 


Fine 

0.14 


Arts 

0.36 


0.40 


0.28 


0.41 


Engagement 


0.17 


1.00 


0.05 


0.10 


0.27 


■0.08 


0.00 


-0.08 


■0.02 


Disengagement 


-0.26 


0.05 


1.00 


0.02 


■0.07 


- 0.19 


■0.01 


0.07 


■0.01 


Quantitative Professions 


-0.11 


0.10 


0.02 


1.00 


0.04 


- 0.10 


- 0.16 


- 0.02 


■0.08 


Academic Time 


0.14 


0.27 


- 0.07 


0.04 


1.00 


0.04 


- 0.04 


- 0.03 


■0.02 


HS GPA 


0.36 


- 0.08 


- 0.19 


- 0.10 


0.04 


1.00 


0.23 


0.19 


0.23 


SAT Reading/ SAT1 Verbal 


0.40 


0.00 


-0.01 


-0.16 


- 0.04 


0.23 


1.00 


0.54 


0.77 


SAT Math/ SAT1 Math 


0.28 


-0.08 


0.07 


-0.02 


■0.03 


0.19 


0.54 


1.00 


0.51 


SAT Writing/ SAT2 Writing 


0.41 


-0.02 


-0.01 


-0.08 


■0.02 


0.23 


0.77 


0.51 


1.00 


GPA 


1.00 


0.18 


-0.28 


Letters and Foreign Languages 
-0.19 0.10 0.47 0.47 


0.34 


0.54 


Engagement 


0.18 


1.00 


-0.06 


0.18 


0.18 


0.01 


0.02 


-0.05 


0.06 


Disengagement 


-0.28 


-0.06 


1.00 


0.02 


■0.21 


- 0.18 


0.05 


0.08 


■0.02 


Quantitative Professions 


-0.19 


0.18 


0.02 


1.00 


0.14 


- 0.14 


- 0.24 


- 0.20 


■0.25 


Academic Time 


0.10 


0.18 


- 0.21 


0.14 


1.00 


0.09 


- 0.08 


- 0.03 


■0.05 


HS GPA 


0.47 


0.01 


- 0.18 


- 0.14 


0.09 


1.00 


0.27 


0.31 


0.33 


SAT Reading/ SAT1 Verbal 


0.47 


0.02 


0.05 


-0.24 


- 0.08 


0.27 


1.00 


0.60 


0.81 


SAT Math/ SAT1 Math 


0.34 


-0.05 


0.08 


-0.20 


■0.03 


0.31 


0.60 


1.00 


0.60 


SAT Writing/ SAT2 Writing 


0.54 


0.06 


-0.02 


-0.25 


■0.05 


0.33 


0.81 


0.60 


1.00 


GPA 


1.00 


0.21 


-0.14 


Mathematics 
-0.01 0.09 


and Science 
0.44 0.36 


0.37 


0.37 


Engagement 


0.21 


1.00 


0.02 


0.14 


0.22 


0.02 


0.02 


-0.03 


0.03 


Disengagement 


-0.14 


0.02 


1.00 


0.01 


■0.13 


- 0.11 


0.07 


0.12 


0.05 


Quantitative Professions 


-0.01 


0.14 


0.01 


1.00 


0.06 


- 0.08 


- 0.18 


- 0.11 


■0.16 


Academic Time 


0.09 


0.22 


- 0.13 


0.06 


1.00 


0.04 


- 0.08 


- 0.10 


■0.06 


HS GPA 


0.44 


0.02 


- 0.11 


- 0.08 


0.04 


1.00 


0.18 


0.19 


0.23 


SAT Reading/ SAT1 Verbal 


0.36 


0.02 


0.07 


-0.18 


- 0.08 


0.18 


1.00 


0.52 


0.78 


SAT Math/ SAT1 Math 


0.37 


-0.03 


0.12 


-0.11 


■0.10 


0.19 


0.52 


1.00 


0.56 


SAT Writing/ SAT2 Writing 


0.37 


0.03 


0.05 


-0.16 


■0.06 


0.23 


0.78 


0.56 


1.00 


GPA 


1.00 


0.19 


-0.15 


0.03 


Engineering 
0.11 0.37 


0.27 


0.37 


0.32 


Engagement 


0.19 


1.00 


0.08 


0.23 


0.26 


0.00 


■0.02 


-0.02 


■0.02 


Disengagement 


-0.15 


0.08 


1.00 


0.03 


■0.15 


- 0.13 


0.06 


0.08 


0.04 


Quantitative Professions 


0.03 


0.23 


0.03 


1.00 


0.11 


- 0.03 


- 0.12 


- 0.14 


■0.12 


Academic Time 


0.11 


0.26 


- 0.15 


0.11 


1.00 


0.03 


- 0.05 


- 0.04 


■0.05 


HS GPA 


0.37 


0.00 


- 0.13 


- 0.03 


0.03 


1.00 


0.21 


0.20 


0.25 


SAT Reading/ SAT1 Verbal 


0.27 


- 0.02 


0.06 


-0.12 


- 0.05 


0.21 


1.00 


0.47 


0.78 


SAT Math/ SAT1 Math 


0.37 


- 0.02 


0.08 


-0.14 


■0.04 


0.20 


0.47 


1.00 


0.55 


SAT Writing/ SAT2 Writing 


0.32 


-0.02 


0.04 


-0.12 


■0.05 


0.25 


0.78 


0.55 


1.00 


GPA 


1.00 


0.11 


-0.15 


Business and Management 
0.06 0.08 0.38 0.41 


0.40 


0.44 


Engagement 


0.11 


1.00 


0.02 


0.16 


0.27 


■0.07 


■0.09 


-0.14 


■0.13 


Disengagement 


-0.15 


0.02 


1.00 


-0.02 


■0.08 


-0.07 


0.03 


0.13 


0.03 


Quantitative Professions 


0.06 


0.16 


-0.02 


1.00 


0.09 


-0.07 


- 0.12 


- 0.12 


■0.13 


Academic Time 


0.08 


0.27 


- 0.08 


0.09 


1.00 


- 0.04 


- 0.16 


- 0.10 


■0.12 


HS GPA 


0.38 


- 0.07 


- 0.07 


- 0.07 


■0.04 


1.00 


0.27 


0.28 


0.34 


SAT Reading/ SAT1 Verbal 


0.41 


- 0.09 


0.03 


-0.12 


- 0.16 


0.27 


1.00 


0.60 


0.82 


SAT Math/ SAT1 Math 


0.40 


-0.14 


0.13 


-0.12 


■0.10 


0.28 


0.60 


1.00 


0.64 


SAT Writing/ SAT2 Writing 


0.44 


-0.13 


0.03 


-0.13 


■0.12 


0.34 


0.82 


0.64 


1.00 
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Discussion 

It is in our nature to see that for which we look. If we use sample-based surveys we see 
institutional patterns with perhaps a few demographic distinctions that can be supported by the 
sample size. If our admissions studies focus on that critical first year, then we see important 
associations between high school requirements, performance in high school, and on admission 
tests. If we redirect our focus on upper-division performance by academic major and if we 
include all students in our campus surveys, we realize that we haven’t been seeing very much 
at all. That is unfortunate because quality of academic programs should be our educational 
focus. It is the quality of our academic programs that most concerns our faculty, parents, 
students, and employers. 

This research takes a step in the direction of supporting admissions within a framework that 
recognizes the disciplinary differences in student experience by answering three questions. 

• Should we view admissions policies as predicting general education outcomes or 
disciplinary specific outcomes? 

• Can we aggregate our 100+ programs per campus to produce a manageable and 
accurate number of clusters? 

• Can we identify the factors associated with performance by students in those clusters? 

While recognizing that admission policies perform many important institutional functions, this 
study was concerned with the academic predictive validity of admissions practices generally and 
more specifically with the nature of the dependent variable. Based on several bodies of 
research and the results of SERU/UCUES census undergraduate surveys, the paper argues 
that general education is a misleading term and that admission standards designed to predict 
success in general education will therefore be misled. Instead, this paper asserts that admission 
policies should be designed to predict outcomes in the academic disciplines. Unfortunately, 
many institutions offer well over 100 academic programs and many students are unclear about 
their major at the time of application. 

Cluster analysis of factor scores from a comprehensive undergraduate census survey was used 
to identify nine broader disciplinary areas: Social Sciences, Agriculture, Fine Arts, Humanities, 
Public Administration, Architecture and Environmental Design, Science and Mathematics, 
Engineering, and Business and Management. These disciplinary areas evidence varying levels 
of relative importance of factors including Engagement, Disengagement, Development of 
Scholarship, Quantitative Professions, and Non-quantitative Skills and Gains. The academic 
clusters are asserted as a minimum level of distinction that should be recognized in admissions. 

A stepwise regression technique (SAS GLMSELECT) was employed to identify the relative 
importance of a variety of admissions, demographic, and survey-based educational experience 
factors. The results were telling. Traditional predictors were important in predicting cumulative 
grade point average of upper-division majors but the measures reflected disciplinary content. 
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and the traditional measures were joined by factor scores from the SERU/UCUES 
questionnaire, especially student Engagement and Disengagement. 

• High school grade point average was an important early predictor for all disciplinary 
areas. That is reasonable given the required broad and rigorous high school curriculum 
for University of California admissions (History/Social Science - 2 years required; 
English - 4 years required; Mathematics - 3 years required, 4 years recommended; 
Laboratory Science - 2 years required, 3 years recommended; Language Other than 
English - 2 years required, 3 years recommended; Visual and Performing Arts - 1 year 
required; and one additional College-Preparatory Elective). 

• The SAT scores were also reasonably identified as predictors and usually in expected 
orders. For Social Sciences, SAT Writing/SAT2 Writing and SAT Reading/SATI Verbal 
were important with Writing appearing first. In Agriculture, SAT Reading/SATI Verbal 
was the first test score to appear and was joined by SAT Math/SATI Math. Fine Arts 
saw only SAT Writing/SAT2 Writing and it appeared first. Test scores were not 
especially important for Letters and Foreign Language but SAT Reading/SATI Verbal 
was added and Math was not. Mathematics and Science saw the broadest range and 
longest list of predictors with SAT Math/SATI Math appearing before SAT 
Reading/SATI Verbal and SAT Writing/SAT2 Writing was well down the series. 
Engineering was best predicted by SAT Math/SATI Math and then SAT Writing/SAT2 
Writing. Business Management saw SAT Writing/SAT2 Writing appearing before SAT 
Math/SATI Math. 

• In all disciplinary areas, questionnaire-based factor scores appeared as predictors. In all 
but one case. Engagement was the first questionnaire-based factor to appear and was 
joined by Disengagement. The Quantitative Professions factor was a predictor in 
Mathematics and Science, Engineering, and Business Management. Current Skills and 
Skill Gains (nonquatitative) were predictors for Social Sciences, Mathematics and 
Science, and Current Skills was a predictor for Engineering. Development of Scholarship 
was last to appear as a significant predictor for Mathematics and Science, and 
Engineering. Demographic variables were inconsistent predictors and typically appeared 
late in the listings with one notable exception. In Letters and Foreign Language, first 
language was the first predictor and there was a GPA advantage for native English 
speakers. 

Whether questionnaire-based factor scores should be considered predictors or outcomes is an 
interesting question that warrants further research. Many of the factors are probably both 
predictors and outcomes, especially Engagement. From this study, it cannot be determined 
whether students earn higher grades by engagement activities, become increasingly engaged 
because they have been academically successful, or that academic success is due in part to 
rewarded past engagement. 

When Engagement was treated as a dependent measure, demographic variables were more 
common predictors than SAT scores or HS grade point average and no combination of 
variables predicted more than 3% of Engagement variance. Engagement as an upper-division 
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student was not highly correlated with academic performance in high school. This suggests that 
Engagement brings something more to the admissions process than is offered by the SAT or 
high school grade point average. It certainly warrants further study, and we will soon have 
helpful information. In the fall of 2008, incoming freshmen at the Berkeley campus answered the 
Engagement factor items about their high school experience and longitudinal study of this group 
will help to identify causal directions 
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Appendix 1: Principal Components and Items 



Factor 2: Current Skills Self-Assessment (Nonquantitative) 

2. Please rate your level of proficiency In the following areas when you started at this campus and now. 

Current ability level 

Analytical and critical thinking skills 
Ability to be clear and effective when writing 
Ability to read and comprehend academic material 
Understanding of a specific field of study 
Ability to speak clearly and effectively In English 

Understanding International perspectives (economic, political, social, cultural etc.) 

Leadership skills 
Computer skills 
Internet skills 
Library research skills 
Other research skills 

Ability to prepare and make a presentation 
Interpersonal (social) skills 

3. Similarly, please rate your abilities now and when you first began at this university on the following dimensions. 

Current ability level 

Ability to appreciate, tolerate and understand racial and ethnic diversity 
Ability to appreciate the fine arts (e.g., painting, music, drama, dance) 

Ability to appreciate cultural and global diversity 
Understanding the Importance of personal social responsibility 
Self awareness and understanding 

Factor 3: Engagement with Studies 

5. How frequently during this academic year have you done each of the following? 

Sought academic help from Instructor or tutor when needed 

Worked on class projects or studied as a group with other classmates outside of class 
Helped a classmate better understand the course material when studying together 

6. How frequently have you engaged In these activities so far this academic year? 

Taken a small research-oriented seminar with faculty 
Communicated with a faculty member by email or In person 

Talked with the Instructor outside of class about Issues and concepts derived from a course 
Interacted with faculty during lecture class sessions 

Worked with a faculty member on an activity other than coursework (e.g., student organization, campus 
committee, cultural activity) 

7. During this academic year, how often have you done each of the following? 

Contributed to a class discussion 

Brought up Ideas or concepts from different courses during class discussions 
Asked an Insightful question In class 

Found a course so Interesting that you did more work than was required 

Chosen challenging courses, when possible, even though you might lower your GPA by doing so 
Made a class presentation 

Had a class In which the professor knew or learned your name 
12. Indicate the following research and creative activities that you are currently doing or have completed as a UC student. 
At least one student research course (e.g., course 99) 

At least one Independent study course (e.g., 199) 

Assist faculty In research with course credit 
Assist faculty In research for pay without course credit 
Assist faculty In research as a volunteer without course credit 
Work on creative projects under the direction of faculty with course credit 
Work on creative projects under the direction of faculty for pay without course credit 
Work on creative projects under the direction of faculty as a volunteer without course credit 
20. How many professors do you know well enough to ask for a letter of recommendation In support of an application for a 
job or for graduate or professional school? 
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Appendix 1: Principal Components and Items (Continued) 

Factor 4: Gains in Self-Assessment of Skiils (Nonquantitative) - Computed change between initiai and current skiil ievels 
for items comprising Factor 2 

Factor 5: Development of Scholarship 

5. How frequently during this academic year have you done each of the following? 

Raised your standard for acceptable effort due to the high standards of a faculty member 
Extensively revised a paper at least once before submitting it to be graded 

16. Thinking back over your coursework this academic year, how often were you REQUIRED to do the following? 

Recognize or recall specific facts, terms and concepts 

Explain methods, ideas, or concepts and use them to solve problems 

Break down material into component parts or arguments into assumptions to see the basis for different 
outcomes and conclusions 

Judge the value of information, ideas, actions and conclusions based on the soundness of sources, methods 
and reasoning 

Create or generate new ideas, products or ways of understanding 

17. Thinking back on this academic year, how often have you done each of the following? 

Used facts and examples to support your viewpoint 

Incorporated ideas or concepts from different courses when completing assignments 

Examined how others gathered and interpreted data and assessed the soundness of their conclusions 

Reconsidered your own position on a topic after assessing the arguments of others 

Factor 7: Academic Disengagement 

5. How frequently during this academic year have you done each of the following? 

Turned in a course assignment late 

Gone to class without completing assigned reading 

Gone to class unprepared 

Skipped class 

15. Were the following factors very important to you in deciding on your major? 

Easy requirements 
Allows time for other activities 

21. You told us earlier how much time you spend studying and working in a week. How many hours do you spend on each 
of these other activities in a typical 7 day week? 

Attending movies, concerts, sports, or other entertainment events 
Participating in physical exercise, recreational sports, or physically active hobbies 
Participating in student clubs or organizations 

Pursuing a recreational or creative interest (arts/crafts, reading, music, hobbies, etc.) 

Socializing with friends 
Partying 

Using the computer for non-academic purposes (games, shopping, email/instant messaging, etc.) 

Watching TV 

Factor 8: Quantitative Professions 

2. Please rate your level of proficiency in the following areas when you started at this campus and now. 

Current ability level 

Quantitative (mathematical and statistical) skills 
Change between self-reported current skill level and skill level at entry 
Quantitative (mathematical and statistical) skills 
15. Were the following factors very important to you in deciding on your major? 

Leads to a high paying job 
Prepares me for a fulfilling career 
Prestige 

Factor TB: Academic Time 

1. During your TYPICAL 7-day (168 hour) week during the academic term, how many hours do you spend doing the 
following? 

Attending classes, discussion sections or labs 
Studying and other academic activities outside of class 
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